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FOREWORD 


This  research  and  development  was  conducted  in  support  of  Project  ZF57-525-001- 
022-03.06  (Improving  Human-Computer  Interaction  for  Command  and  Control  Systems) 
under  the  sponsorship  of  the  Naval  Sea  Systems  Command.  The  objective  of  this  project 
is  to  enhance  the  effectiveness  of  command  and  control  systems  through  improved  design 
of  the  human-computer  interface.  In  particular,  the  project  is  designed  to  examine  the 
impact  of  information  overload  on  threat  analysis  performance  in  anti-air  warfare  and  to 
recommend  procedures  for  reducing  the  impact  of  such  overload  on  operational  readiness. 

This  report  documents  the  initial  effort  toward  quantifying  limits  in  human  informa¬ 
tion  processing  that  are  associated  with  critical  command  and  control  operations.  The 
research  vehicle  reported  here  will  provide  the  basis  for  subsequent  investigations  of 
human  decision  and  information  processing  behavior. 

Appreciation  is  expressed  to  EMC  Jones  of  the  Naval  Training  Center,  San  Diego,  and 
to  Dr.  Carl  Englund  of  the  Naval  Health  Research  Center,  San  Diego,  for  their  assistance 
in  providing  research  subjects. 


JAMES  F.  KELLY,  JR. 
Commanding  Officer 


JAMES  J.  REGAN 
Technical  Director 
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SUMMARY 


Problem 


The  analysis  of  threat  is  a  critical  function  in  many  combat  systems,  especially  under 
the  conditions  of  high  information  load  that  typify  anti-air  warfare  (AAW)  operations.  To 
reduce  the  effect  of  information  overload  on  AAW  operator  performance,  it  is  necessary 
to  understand  more  precisely  the  relationship  between  task  load  and  decision  per¬ 
formance.  Proper  investigation  of  this  relationship  requires  the  development  of  a 
research  vehicle  that  embodies  major  features  of  the  AAW  threat  analysis  problem  and 
permits  detailed  quantitative  measurement  of  human  performance. 

Objective 

The  objective  of  this  effort  was  to  develop  and  test  a  system  for  investigating  human 
performance  in  the  conduct  of  AAW  threat  analysis.  The  system  had  to  be  flexible,  be 
portable,  require  minimal  training,  induce  a  high  motivation  to  perform,  and  provide 
detailed  measurement  of  human  performance. 

Approach 

A  simulation  of  the  AAW  threat  analysis  problem  was  developed  and  embedded  in  an 
interactive  "air  defense  game"  driven  by  a  Tektronix  4051  microcomputer.  The  game 
simulated  hostile  air  targets  approaching  the  player's  ship  at  one  of  three  speeds.  The 
entire  scenario  was  displayed  on  the  computer's  CRT  screen.  Task  difficulty  was 
manipulated  under  computer  control  by  varying  the  number  of  targets  and  their  arrival 
times.  The  player  defended  the  ship  by  launching  "missiles"  at  the  incoming  targets.  The 
ship's  detection  range  exceeded  the  range  of  its  missiles,  however,  so  that  a  launch-time 
decision  was  necessary  for  each  target.  The  goal  for  the  player  was  to  kill  all  threats  at 
maximum  range,  but  missiles  launched  too  early  would  fall  short  of  the  target  and  be 
ineffective. 

Following  each  engagement,  the  computer  program  gave  performance  feedback  to 
the  player  and  stored  all  relevant  data  for  subsequent  review  by  the  experimenter.  A  data 
analysis  program  was  developed  to  provide  details  of  player  performance. 

Seventeen  Navy  enlisted  men  served  in  an  initial  experiment  to  evaluate  the  system's 
utility  for  human  performance  research.  Task  variables  were  the  number  of  targets  in 
each  engagement,  the  pace  or  tempo  of  operations,  and  target  speed.  The  effects  of  two 
different  training  sequences  and  of  a  concurrent  auditory  monitoring  task  were  also 
investigated. 

Findings 


1.  The  feasibility  of  the  software  concept  and  its  implementation  were  confirmed 
by  the  preliminary  experiment. 

2.  On-line  extraction  of  performance  data  permitted  researchers  to  compare  the 
performance  of  the  players  with  that  of  a  mathematically  ideal  information  processor. 

3.  Test  subjects  became  proficient  after  about  3  hours  of  practice.  Training 
manipulations  had  little  effect  on  the  course  of  skill  acquisition. 
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4.  The  effects  of  task  load  were  evidenced  by  a  decline  in  performance  as  the 
number  of  targets  and  the  pace  of  operations  were  increased.  Performance  was  also 
impaired  by  the  introduction  of  a  concurrent  auditory  monitoring  task. 

5.  The  air  defense  game  was  challenging  and  induced  a  high  level  of  motivation. 
Test  subjects  sustained  their  attention  to  the  task  for  3  to  4  hours  at  a  sitting. 

Conclusions 


The  air  defense  game  offers  a  rich  analogue  of  AAW  threat  operations.  It  is  readily 
learned,  motivating  to  perform,  and  provides  an  effective  vehicle  for  human  performance 
research. 

Recommendations 


1.  The  air  defense  game  should  be  exploited  as  a  tool  in  future  research  on  the 
threat  analysis  problem. 

2.  The  air  defense  game  should  be  used  to  determine  limitations  in  human 
information  processing  and  to  find  out  how  threat  analysis  strategies  change  as  a  function 
of  task  load. 

3.  Performance  in  the  air  defense  game  should  be  used  as  a  dependent  measure  in 
other  military  research  applications  (e.g.,  the  effect  of  sleep  loss  or  extended  effort). 
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INTRODUCTION 


Problem 


The  analysis  of  threat  is  a  critical  function  in  many  combat  systems,  especially  under 
the  conditions  of  high  information  load  that  typify  anti-air  warfare  (AAW)  operations.  In 
order  to  reduce  the  effect  of  information  overload  on  AAW  operator  performance,  it  is 
necessary  to  understand  more  precisely  the  relationship  between  task  load  and  decision 
performance.  Proper  investigation  of  this  relationship  requires  the  development  of  a 
research  vehicle  that  embodies  major  features  of  the  AAW  threat  analysis  problem  and 
permits  detailed  quantitative  measurement  of  human  performance. 

Objective 

The  objective  of  this  work  was  to  develop  and  test  a  system  for  investigating  human 
performance  that  would: 

•  Exercise  the  complex  cognitive  processes  encountered  in  AAW  threat  analysis 
operations. 

•  Require  minimal  training. 

•  Make  minimal  psychomotor  demands. 

•  Enable  detailed  performance  measurements. 

•  Have  a  military  character  and  be  inherently  motivating  to  perform. 

•  Use  a  single  stand-alone  microcomputer. 

•  Be  flexible  enough  to  enable  the  use  of  varied  scenarios. 

•  Permit  ready  evaluation  of  its  operation. 

Background 

As  the  naval  tactical  environment  becomes  more  complex  and  fast-paced,  the  need 
for  timely  and  effective  decisions  imposes  an  increasingly  heavy  burden  on  command. 
Human  performance  and  the  design  of  the  human-computer  interface  become  critical. 
Thus,  it  is  imperative  that  combat  systems  be  designed  to  minimize  potential  performance 
deficits  under  conditions  of  heavy  information  load.  Proper  guidelines  for  design  should 
include  quantitative  human  performance  data,  as  has  been  emphasized  by  representatives 
of  NAVELEX,  fleet  C2  sites,  and  the  Chief  of  Naval  Operations  (Command,  Control,  and 
Communications  Programs  Office  (OP -94)).  The  Navy  Personnel  Research  and  Develop¬ 
ment  Center  (NAVPERSRANDCEN)  has  an  ongoing  research  effort  that  seeks  to  quantify 
human  performance  limits  in  dealing  with  AAW  threats. 

The  problem  of  information  overload  is  especially  acute  at  the  level  of  the  individual 
ship,  where  the  Combat  Direction  Center  staff  must  integrate  data  from  several  sources 
to  detect,  classify,  and  monitor  contacts.  Staff  personnel  must  attend  concurrently  to 
multiple  contacts  and  make  accurate  and  timely  decisions  under  stress  (Combat  Direction 
Systems  Department  Organizational  Manual,  USS  AMERICA  (CV  66),  1978;  Cullison,  1979; 
Halnon,  1979). 


The  analysis  of  threat  in  tactical  operations  is  largely  a  covert,  cognitive  activity 
about  which  little  is  known.  In  general  terms,  decision  makers  must  recognize  contacts  as 
potential  threats,  set  priorities,  and  initiate  appropriate  actions  to  neutralize  contacts 
determined  to  be  hostile.  A  Ship  Weapons  Coordinator  (SWC),  for  example,  must  respond 
to  messages  and  alerts  from  the  staff,  monitor  the  status  of  potential  threats,  assign  the 
appropriate  weapons,  and  carry  out  directives  from  his  superiors.  This  sequence  works 
reasonably  well  in  routine  situations,  but  problems  occur  when  track  loads  become  heavy, 
as  they  often  do  in  AAW  operations.  Here  the  rate  of  information  flow  increases 
dramatically  and  performance  begins  to  degrade.  The  SWC's  job  is  especially  vulnerable 
to  information  overload. 

A  recurring  problem  in  the  study  of  information  overload  has  been  the  selection  of  a 
measure  of  cognitive  performance  that  is  sensitive  to  variations  in  task  load.  The  more 
successful  studies,  such  as  those  by  North  and  Gopher  (1976)  and  Wickens  and  Gopher 
(1977),  have  addressed  the  aircraft  environment.  Typically,  a  compensatory  tracking  task 
has  been  used  to  simulate  the  psychomotor  demands  of  flying.  The  tracking  task  was 
performed  in  conjunction  with  other  tasks,  such  as  map  reading,  and  tracking  error  was 
then  taken  as  an  indicator  of  mental  workload.  Harris,  North,  and  Owens  (1978)  have 
described  an  experiment  controller  system  that  employs  this  paradigm.  The  specifics  of 
this  dual-task  paradigm  cannot  be  applied  directly  to  AAW,  however,  because  AAW 
problems  place  little  or  no  reliance  on  psychomotor  skills  like  tracking.  Instead,  AAW 
subsystems  require  sustained  attention  to  complex  information  processing  tasks  in  which 
threat  analysis  has  high  priority.  Therefore,  at  the  start  of  this  effort,  a  research  tool 
was  needed  that  would  incorporate  the  cognitive  features  of  AAW  threat  analysis,  provide 
close  experimental  control  of  task  events,  and  yield  detailed  quantitative  measurements 
of  human  performance. 


AIR  DEFENSE  GAME 

A  laboratory  simulation  of  the  AAW  threat  analysis  problem  was  developed  and 
embedded  in  an  "air  defense  game"  driven  by  a  Tektronix  4051  microcomputer.  This 
approach  captured  the  important  human  information  processing  demands  of  AAW  threat 
analysis  by  requiring  sustained  attention  to  a  complex  "hostile"  environment  that  unfolded 
in  real  time  and  responded  appropriately  to  the  actions  of  the  player.  At  the  same  time, 
the  simulation  scheme  met  the  researchers'  requirements  for  experimental  control  and  for 
quantitative  measurements  of  human  performance.  The  computer  is  shown  in  Figure  1. 

Scenario 


The  computer  supervises  an  AAW  scenario  in  which  a  player  defends  a  ship  by 
launching  "missiles"  at  "hostile  targets"  that  appear  on  the  computer's  CRT  screen. 
Instructions  given  to  the  player  appear  in  Figure  2.  The  goal  is  to  kill  all  targets  at 
maximum  range,  but  the  ship's  detection  range  exceeds  its  weapons  range  so  that  a 
launch-time  decision  is  necessary  for  each  target.  Missiles  launched  too  early  fall  short 
of  their  targets  (i.e.,  splash).  It  is  possible  to  fire  again  after  a  splash,  but  only  one 
missile  is  allowed  to  be  in  flight  on  a  given  track  at  one  time.  Duplicate  (inflight) 
launches  result  in  penalties.  Penalties  are  also  incurred,  of  course,  when  targets  hit  the 
ship. 
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hgure  i.  Computer  displaying  the  air  delense  game  in 


AIR  DEFENSE  GAME  INSTRUCTIONS 

YOUR  SHIP  IS  UNDER  ATTACK  BY  INCOMING  MISSILES  WITH 
VARYING  SFEEDS  AND  LAUNCH  TIMES.  YOUR  TASK  IS  TO  MONITOR 
THE  RADAR  DISPLAY  AND  DESTROY  THEM.  THE  PRIORITIES 
OF  THE  ENGAGEMENT  ARE: 

<1>  AVOID  BEING  HIT. 

<2/  AVOID  SPLASHING  YOUR  OWN  MISSILES  BY  LAUNCHING  TOO 
EARLY... YOUR  WEAPONS  RANGE  IS  20  MILES.  WHICH 
IS  THE  INNER  CIRCLE  OF  THE  RADAR  DISPLAY. 

(3T  DESTROY  INCOMING  MISSILES  AS  SOON  AS  POSSIBLE 
AFTER  THEY  ENTER  YOUR  WEAPONS  RANGE. 

(4)  AVOID  LAUNCHING  A  MISSILE  IF  YOU  ALREADY  HAUE  ONE 
IN  PLIGHT  ON  THE  SAME  TARGET. 

YOUR  SKILL  RATING  <-0-100/  WILL  INCLUDE  A  12-POINT 
PENALTY  FOR  EACH  HIT  SUSTAINED  AND  A  2-POINT  PENALTY  FOP 
EACH  INFLIGHT  LAUNCH.  THE  MAXIMUM  KILL  RANGE  IS  20  MILES. 

FIVE  POINTS  APE  DEDUCTED  FOR  EACH  MILE  THAT  YOUR  AUERAGE 
KILL  RANGE  IS  UNDER  20. 

TO  LAUNCH  A  MISSILE,  USE  THE  TEN  WHITE  KEYS  AT  THE  UPPER 
LEFT  OF  THE  KEYBOARD.  ENTER  THE  TWO-DIGIT  TRACK  NUMBER 
OF  THE  TARGET... A  READOUT  WILL  THEN  INFORM  YOU  OF  A  SUCCESSFUL 
LAUNCH— OR  AN  ERROR.  AN  ERROR  OCCURS  IF  YOU  KEY  A  NUMBER 
INCORRECTLY  OR  IF  YOU  LAUNCH  A  MISSILE  UNNECESSARILY. 

GOOD  LUCK _ THE  FATE  OF  YOUR  SHIP  LIES  IN  YOUR  HANDS! 


Figure  2,  CRT  display  of  instructions  for  air  defense  game. 
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Game  Activity  * 

Figure  3  shows  a  sample  snapshot  of  the  game  in  progress.  The  display  simulates  a 
radar  screen  in  which  the  ship  is  the  "+"  sign  at  the  center,  the  inner  circle  is  the  weapons 
range  (20  miles),  and  the  outer  circle  is  the  radar's  detection  range  (46  miles).  (The  radius 
of  the  outer  circle  is  13.5  centimeters.)  Targets  appear  as  dots  (blips)  on  the  display  and 
each  is  randomly  assigned  a  unique  track  number  (TN),  01  through  99,  that  appears  at  the 
outer  circle  as  soon  as  the  target  enters  the  display.  All  targets  head  directly  toward  the 
ship  at  randomly  selected  bearings. 
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The  computer,  simulating  a  radar  sweep,  updates  the  display  every  11.7  seconds,  and 
at  each  update  another  blip  is  displayed  on  all  target  paths.  The  distance  between  blips, 
then,  is  a  cue  to  the  relative  speed  of  the  targets.  In  Figure  3,  TN  24  is  fast,  TN  30  is 
medium,  and  TN  60  is  slow;  their  respective  speeds  are  5,  3,  and  1.6  miles  per  update. 
The  sweep  time  varies  from  0.6  to  0.8  seconds  and  the  time  between  sweeps  is  available 
for  action  by  the  player  and  processing  by  the  computer. 

The  player  launches  missiles  by  pressing  function  keys  at  the  upper  left  of  the 
computer's  keyboard.  Each  keypress  codes  one  digit  of  the  required  2-digit  TN.  Feedback 
for  a  good  launch  (i.e.,  at  a  valid  track)  is  indicated  at  the  upper  left  of  the  display  by 
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flashing  the  keyed  TN  next  to  the  letter  "L"  (for  "launched").  An  illegal  launch  attempt 
(i.e.,  for  a  nonexistent  track  or  if  a  missile  is  already  in  flight  on  the  track)  is  indicated  by 
flashing  the  keyed  TN  next  to  the  letter  "E"  (for  "error"). 


When  the  player  launches  a  missile,  a  dash  (-)  is  immediately  displayed  to  represent 
its  path,  and  another  dash  is  added  at  each  update  thereafter.  Missiles,  like  fast  targets, 
travel  5  miles  per  update.  When  the  missile  reaches  its  intended  target,  a  kill  occurs,  a 
bell  sounds,  and  the  appropriate  radius  is  filled  by  a  solid  line  on  the  display.  If  an 
incoming  target  reaches  the  ship,  a  hit  occurs,  a  longer  bell  sounds,  and  the  TN  of  the 
target  is  blocked  out. 

In  the  example  shown  in  Figure  3,  the  player  has  destroyed  TN  97  and  has  sustained  a 
hit  from  the  target  on  the  track  at  4  o’clock.  A  missile  has  just  been  launched  at  TN  84, 
and  missiles  were  launched  against  TNs  24  and  27  two  sweeps  earlier.  A  splash  has 
occurred  on  TN  25;  that  missile  reached  weapons  range  too  early  to  kill  its  assigned 
target. 

Player  Feedback 

A  single  play  of  the  game  continues  until  each  target  in  the  scenario  has  either  been 
killed  or  has  hit  the  ship.  The  screen  is  then  erased  and  the  player  is  given  feedback  in 
the  form  of  an  air  defense  summary  (see  Figure  4).  The  "average  range  for  kills"  given 
in  the  summary  is  the  mean  distance  from  the  ship  to  the  points  where  targets  were 
destroyed  by  missiles  (hits  are,  of  course,  excluded  from  this  computation).  An  overall 
"skill  rating"  (R)  is  provided  that  takes  each  of  the  engagement  priorities  into  account. 
The  rating  is  defined  as: 

R  =  100  (Average  kill  range/20)  -  12  (No.  of  hits)  -  2  (No.  of  inflights). 


AIR  DEFENSE  SUMMARY 

PLAYER  NO.  10 

HO.  OF  TARGETS  *  IS  TEMPO: 

INTERMEDIATE 

SPEED  #  PILLS  • 

HITS 

FAST  5 

I 

MEDIUM  6 

0 

SLOW  6 

0 

TOTAL  KILLS  *  17  TOTAL  HITS 

=  I 

AVERAGE  RANGE  FOP  KILLS  =■  16.42 

NO.  OF  MISSILES  LAUNCHED  =  IS 

NO.  OF  MISSILES  SPLASHED  =  1 

HO.  OF  INFLIGHT  LAUNCHES  *  0 

(MAX.  =  20) 

SKILL  RATING  *  70  (HAX.  =  100) 

Data  is  being  stored  oti  tape... 

DO  YOU  WANT  TO  PLAY  AGAIN  (YH)' 

Figure  4.  Feedback  information  displayed  at  the 
end  of  each  engagement. 
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If  performance  is  perfect,  all  targets  will  be  killed  at  the  maximum  range  of  20 
miles,  there  will  be  no  hits  or  inflight  launches,  and  R  will  be  100.  While  incurred 
penalties  can  render  R  <  0,  R  =  0  was  the  minimum  value  displayed  as  feedback  to  the 
player. 

Task  Variables 


The  major  variables  affecting  the  conduct  of  the  air  defense  game  are  the  number  of 
targets,  the  tempo  of  operation,  and  target  speed. 

Number  of  Targets 

The  size  of  the  computer's  memory  limits  the  number  of  targets  presented  in  any 
engagement  to  72.  A  displayed  menu  makes  five  selections  available:  6  (for  practice 
only),  18,  36,  54,  or  72.  Equal  numbers  of  fast,  medium,  and  slow  targets  are  always 
assigned.  The  minimum  separation  between  tracks  is  4°. 

Tempo  of  Operation 

The  tempo  is  also  selected  from  a  menu  and  controls,  through  one  of  three 
randomization  routines,  the  rate  at  which  the  targets  appear.  In  the  low  tempo,  target 
entry  times  are  relatively  spread  out  and  prosecuting  the  threats  is  comparatively  easy. 
As  the  tempo  is  raised,  the  information  processing  demands  increase.  Entry  time 
patterns,  though  varying  with  tempo  and  from  engagement  to  engagement,  always  have 
the  following  characteristics: 

•  Slow  targets,  which  require  the  longest  time  to  reach  the  ship,  generally  enter 
early  in  the  engagement. 

•  Medium-speed  targets  enter  most  prominently  during  the  middle  of  the  engage¬ 
ment. 

•  Fast  targets  have  their  entries  distributed  over  the  entire  engagement,  except 
for  their  absence  in  the  first  several  updates. 

For  any  fixed  number  of  targets,  increasing  the  tempo  increasingly  crowds  the  times 
of  entry.  The  effect  is  to  shorten  the  engagement  and  yet  preserve  the  patterns  of  entry 
times  just  described.  This  can  be  seen  in  Figure  5,  which  depicts  the  entry-time 
distributions  of  18  targets  for  the  three  tempos.  Each  horizontal  axis  is  the  sequence  of 
display  updates,  and  the  letters  F,  M,  and  S  (six  of  each)  indicate  the  entry  of  fast, 
medium,  and  slow  targets.  Each  entry  is  stochastic  (i.e.,  each  target  is  equally  likely  to 
enter  at  its  indicated  update  U  or  at  U  +  1).  In  effect,  a  fair  coin  is  tossed  separately  for 
each  of  the  18  targets  to  determine  its  actual  entry  time  in  the  given  engagement. 

The  pattern  of  entry  times  given  in  Figure  5  does  not  imply  a  similar  pattern  for 
missile  launches.  Targets  entering  the  display  at  different  times  with  different  speeds 
may  very  well  enter  the  ship's  missile  range  at  the  same  time.  The  effect  of  each 
schedule  is  to  create  a  peak  demand  for  missile  launches  during  the  middle  of  each 
engagement. 
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For  any  engagement,  the  number  of  targets  and  the  tempo  combine  to  determine  the 
number  of  updates  to  the  entry  of  the  last  target,  the  mean  number  of  updates  between 
successive  target  arrivals,  and  the  approximate  duration  of  the  engagement.  The  actual 
duration  depends  in  part  upon  the  player's  proficiency  in  responding  to  those  tracks  that 
appear  near  the  end  of  the  game.  Engagement  characteristics  are  summarized  in  Table  1. 


Target  Speed 


As  previously  stated,  the  three  target  speeds  are  5.0,  3.0,  and  1.6  miles  per  update, 
with  initial  ranges  at  entry  being  45,  44,  and  44  miles  respectively.  If  the  player  fails  to 
fire,  a  fast  target  will  hit  the  ship  at  its  10th  update;  a  medium  target,  at  its  16th  update; 
and  a  slow  target,  at  its  28th  update. 


Performance  Measurement  and  Analysis 


A  major  advantage  of  on-line  computerized  control  is  that  details  of  the  player's 
behavior  can  be  sensed  and  stored  automatically.  Here,  each  missile  launch  by  the  player 
is  stored  in  memory  and  identified  with  its  track  number  and  the  display  update  on  which 
it  occurred.  The  parameters  and  schedules  of  the  targets  also  reside  in  memory.  These 
data  are  stored  on  magnetic  tape  at  the  end  of  each  game  so  that  the  engagement  can  be 
reconstructed  for  the  experimenter  and  analyzed  in  detail. 


A  separate  off-line  analysis  program  retrieves  the  data  from  tape  and  displays  an 
overall  performance  summary  identical  to  that  shown  in  Figure  4.  Additional  analyses 
provide  the  experimenter  with  more  detailed  summaries  of  the  engagement.  Fundamental 
to  these  analyses  is  the  notion  that  there  is  an  optimal  launch  time  that  can  be  used  as  a 
standard  for  player  performance. 
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Table  1 


Engagement  Characteristics  as  Determined  by 
the  Number  of  Targets  and  the  Tempo  of  Operation 


Number 

of 

Targets 

Tempo  of  Operation 

Low 

Intermediate 

High 

Number  of  Updates  to  Last  Target  Entrya 

18 

30 

19 

14 

36 

60 

34 

24 

54 

90 

49 

34 

72 

120 

64 

44 

Mean  Number  of  Updates  Between  Successive  Target  Arrivals 


IS 

1.70 

1 .06 

.76 

36 

1.68 

.94 

.66 

54 

1.68 

.90 

.62 

72 

1.68 

.89 

.60 

Duration  of  Engagement  in  Minutes*3 

18 

7.2 

5.0 

4.2 

36 

13.0 

8.0 

6.0 

54 

18.5 

10.8 

8.0 

72 

24.8 

14.2 

9.9 

Note;  Updates  occur  at  intervals  of  approximately  11.7  seconds.  The  precise  interval 
depends  upon  the  status  of  the  targets  and  the  actions  of  the  player. 

aThe  randomization  procedure,  with  probability  =  '/z,  adds  one  to  the  tabled  value. 

^These  are  approximate  playing  times  for  a  single  engagement.  Actual  durations  vary 
somewhat  with  random  scheduling  effects  and  the  proficiency  of  the  player. 


Optimal  Launch  Time 

If  player  performance  is  optimal,  each  target  will  be  destroyed  at  precisely  the  20 
mile  weapons  range  circle.  For  a  fast  target  entering  the  display  at  update  E,  the  optimal 
launch  time  (L*)  is  at  update  E  +  2,  which  corresponds  to  the  display  of  its  third  blip.  For 
a  medium  target  L*  =  E  +  5,  its  sixth  blip.  Similarly,  for  slow  targets,  we  have 
L*  =  E  +  12. 
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As  one  performance  measure,  the  player's  actions  are  compared  with  the  optimal 
tactics  by  taking  the  difference  between  observed  launch  time  (L)  and  optimal  launch 
time  (L*)  to  yield  the  lag  =  L  -  L*.  The  ideal  player  will  have  zero  lags  for  all  targets, 
and  increasing  lags  generally  represent  poorer  performance  by  the  player.  If  a  target  hits 
the  ship,  we  arbitrarily  assign  a  lag  =  -  L*.  Firing  too  early  (a  splash)  would  also  result  in 
a  lag  <  0,  but  subsequent  behavior  by  the  player- -either  a  kill  or  allowing  a  hit  to 
occur--determines  the  actual  lag  assigned  to  that  target. 

Target  Information  and  Weapons  Employment  Summary 

Figure  6  shows  an  example  of  a  target  information  and  weapons  summary.  For  each 
of  the  18  targets  in  this  engagement,  the  top  part  of  the  display  lists  the  target  speed  (F, 


M,  or  S)  and  the  following: 

•  Bearing  angle  in  degrees . ANG 

•  Track  number  (01  through  99) .  TN 

•  Entry  time  (update  number) .  E 

•  Optimal  launch  time  (update  number) .  L* 

•  Actual  launch  time  (update  number) .  L 


ANG 

TN 

E 

L* 

KL 

LAG 

IN 

SP 

0 

1 

F 

112 

46 

5 

-» 

< 

0 

_  -> 

*  ( 

0 

0 

H 

2 

F 

328 

24 

10 

12 

12 

0 

0 

0 

K 

3 

F 

336 

10 

16 

18 

19 

1 

0 

0 

y 

4 

4 

27 

10 

12 

12 

0 

0 

0 

K 

5 

F 

140 

18 

14 

16 

IS 

2 

0 

0 

K 

6 

F 

176 

58 

20 

2.2 

22 

0 

0 

0 

K 

< 

M 

36 

21 

4 

"9 

15 

6 

0 

0 

V 

8 

M 

284 

30 

8 

13 

15 

2 

0 

0 

K 

0 

M 

72 

53 

12 

17 

18 

1 

0 

0 

K 

10 

M 

288 

9? 

5 

10 

10 

0 

0 

0 

f. 

11 

M 

172 

26 

10 

15 

18 

3 

0 

0 

K 

12 

M 

8 

41 

13 

18 

19 

1 

0 

0 

K 

13 

S 

12 

63 

1 

13 

16 

3 

0 

0 

K 

14 

s 

40 

25 

4 

16 

17 

1 

0 

1 

K 

15 

s 

349 

60 

5 

17 

19 

2 

0 

0 

K 

18 

s 

240 

84 

1 

13 

14 

1 

0 

0 

K 

1? 

s 

69 

47 

2 

14 

19 

5 

0 

0 

K 

18 

s 

184 

55 

4 

16 

18 

2 

0 

0 

K 

MEAN  LAG 

C 

o 

v  LAG 

H 

IN 

SP 

FAST 

TARGET 

c 

w* 

0.60 

0.80 

5 

0 

0 

MED 

TARGET 

c 

2.17 

1.95 

6 

0 

0 

SLOW 

TARGETS 

2.33 

1.37 

6 

0 

1 

ALL 

TARGET 

c 

1.76 

1.66 

17 

0 

1 

Figure  6.  Summary  of  target  information  and  weapons 
employment  for  an  18-target  engagement. 
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LAG 


•  Lag  time  (L  -  L*)  .... 

•  Number  of  inflight  launches  .  IN 

•  Number  of  splashes .  SP 

•  Outcome  (K  =  target  killed,  H  =  hit  on  ship) .  O 


The  bottom  of  the  display  shows  the  mean  lags  and  standard  deviations  (SD  LAG)  for 
each  target  speed  and  for  all  targets  combined.  The  sample  size  (N)  for  these  statistics 
includes  only  those  targets  that  are  killed  by  the  player.  Finally,  the  number  of  inflight 
launches  (IN)  and  splashes  (SP)  are  totaled  for  each  target  spead. 

Dynamic  Performance  Summary 

The  above  analyses  refer  to  individual  targets  or  to  averages  over  targets  of  a  given 
speed.  As  such,  they  do  not  capture  the  player's  performance  as  the  engagement  unfolds 
in  time.  To  obtain  this  kind  of  dynamic  profile,  the  analysis  program  summarizes  the 
situation  at  each  update  of  the  engagement.  Figure  7  shows  such  a  summary  for  an  18- 
target  engagement  concluded  in  25  updates.  The  summary  lists  the  following  data: 


•  Update  number .  lj 

•  Number  of  active  targets .  ACT 

•  Number  of  targets  engaged  by  missiles .  UNG 

•  Cumulative  number  of  targets  killed .  K 

•  Cumulative  number  of  hits  on  ship .  H 

•  Number  of  targets  that  remain  scheduled  to  appear .  REM 

•  Number  of  unengaged  targets  within  firing  range .  UT 

•  Number  of  missed  opportunities  to  fire  at  eligible  targets .  MO 


The  UT  and  MO  scores  are  candidate  measures  that  seek  to  capture,  at  each  update, 
the  extent  to  which  the  player's  actions  are  failing  behind  optimal  performance.  The  UT 
measure  is  simply  the  number  of  eligible  targets  that  have  yet  to  be  fired  on  at  the 
current  update. 

Note  that  the  UT  measure  does  not  consider  the  number  of  prior  updates  on  which  a 
target  has  been  eligible.  The  alternative  measure  MO  includes  this  history.  It  is  derived 
as  follows:  For  the  current  update,  denote  the  UT  unengaged  targets  within  range  as 
T1,”mTUT*  suc^  tar8et*  Tj,  has  been  in  this  state  for,  say  Xj  updates  including  the 

current  one.  The  measure  MO  is  then  the  sum  of  the  x;  for  i  =  1,...,UT.  Thus,  it  should  be 

seen  that  MO  is  the  number  of  missed  opportunities  to  fire  at  eligible  but  unengaged 
targets.  Equivalently,  for  any  snapshot  of  the  display  at  a  given  update,  MO  equals  the 
total  number  of  displayed  blips  within  firing  range  for  unengaged  targets. 
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Figure  7.  Dynamic  performance  summary. 


MO  and  UT,  then,  provide  related  measures  of  the  player's  efficiency  as  the 
engagement  unfolds  over  time.  It  should  be  clear  that  MO  >  UT  and  that,  for  an  ideal 
player  who  fires  at  the  optimal  launch  time,  MO  =  UT  =  0  for  all  updates.  Real  players, 
however,  cannot  process  all  targets  as  quickly  as  they  become  eligible. 

A  profile  of  MO  and  UT  is  displayed  by  the  analysis  program,  as  shown  in  Figure  8. 
For  the  UT  index  (the  "+"  signs),  the  first  target  came  into  firing  range  at  the  7th  update; 
the  player  did  not  launch  a  missile  at  that  time,  so  UT  =  1  for  update  7.  There  were  two 
eligible  but  unengaged  targets  on  the  updates  9  through  12,  etc.  As  illustrated  by  the  UT 
profile,  this  player  progressively,  and  typically,  fell  behind  optimal  performance  during 
the  higher-paced  middle  portion  of  the  engagement.  The  MO  index  (the  solid  line)  reveals 
a  similar  impairment  of  performance  during  the  period  of  peak  processing  demand.  The 
asterisk  in  Figure  8  indicates  that  a  hit  was  sustained  on  the  14th  update. 


A*. 
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=  number  of  unengaged  targets  within  firing  range. 

=  number  of  missed  opportunities  to  fire  at  such  targets. 
=  first  target  enters  firing  range. 

=  hit(s)  sustained  at  this  update. 


Figure  8.  Two  performance  indices  as  functions  of  update. 


Hardware  Characteristics 


A  Tektronix  4051  microcomputer  controls  the  air  defense  simulation  and  the 
subsequent  analyses  of  player  performance.  It  has  32K  bytes  of  memory,  a  direct-view 
storage  tube  display  (19.1  cm  by  14.0  cm),  and  a  magnetic  tape  cartridge  drive  for  storage 
of  programs  and  data  files.  The  programming  language  is  Graphic  System  BASIC 
(Tektronix,  1976).  The  speed  of  graphic  and  arithmetic  operations  was  enhanced  by 
attaching  an  FP-51  Read-Only-Memory  Pack.1 

The  keyboard  has  the  standard  typewriter  layout,  an  auxiliary  numeric  entry  keyset, 
and  10  user-definable  function  keys  at  the  upper  left.  These  latter  were  numbered  from 
0-9  and  were  used  to  enter  track  numbers  to  launch  missiles.  The  function  keys  were  the 
only  ones  used  during  engagements;  a  Plexiglas  sheet  covered  the  main  keyboard  to 
prevent  inadvertent  key  presses. 

Program  Flow 

The  program  listing  for  the  air  defense  game  appears  in  Appendix  A;  a  list  and 
description  of  variables  is  given  in  Appendix  B,  The  program  flow  for  the  game  is  shown 
in  Figure  9.  The  first  part  of  the  program  (START  to  node  B  in  the  flowchart)  provides 
instructions  to  the  player  and  sets  up  the  engagement  parameters.  The  major  section 
(nodes  B  to  G)  controls,  for  each  update,  the  scheduled  entry  and  display2  of  targets  and 
missiles,  tallies  the  player's  actions  and  other  events,  and  generally  supervises  the 
progress  of  the  scenario.  At  the  end  of  the  engagement,  the  last  section  of  the  main 
program  (Figure  9-c)  controls  the  feedback  display  and  the  storage  of  data  on  magnetic 
ytpe.  A  special  weapon-launching  routine  (Figure  9-d)  interrupts  the  main  program  to 
process  missile  firings  as  they  are  keyed  in  by  the  player.  From  the  player's  perspective, 
the  interrupt  software  causes  no  disruption  of  the  flow  of  the  game. 

In  the  off-line  data  analysis  program,  the  program  flow  is  straightforward:  the  user 
selects  the  file  to  be  analyzed  and  sequences  through  the  displayed  output.  The  program 
listing  is  in  Appendix  C;  the  variables  and  their  descriptions  are  in  Appendix  D. 

Limitations 


The  characteristics  of  the  hardware  and  software  impose  the  following  limitations: 

•  Memory  size  limits  the  number  of  targets  to  72. 

•  The  11.7  second  interval  between  updates  of  the  simulated  radar  display  cannot 
be  reduced  while  maintaining  a  constant  inter-update  interval  for  all  combinations  of  the 
task  variables.  If  desired,  the  interval  can  be  increased  by  the  straightforward  insertion 
of  a  dummy  routine. 


‘Micro  Works,  P.O.  Box  1110,  Del  Mar,  California  92014. 

2To  reduce  variability  in  display  timing,  on  every  update  the  program  has  an 
inventory  of  90  targets  that  are  spaced  4°  apart.  Only  the  active  ones  are  displayed;  all 
others  are  suppressed. 


Figure  9a.  Summary  flowchart  of  air  defense  game 


Figure  9b. 
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GAME 


Figure  9d 


•  The  display  is  a  storage  tube  and  cannot  be  selectively  erased;  thus,  the  paths  of 
all  targets  and  missiles  must  remain  on  the  screen  for  the  duration  of  the  engagement. 
This  tends  to  clutter  the  display  when  the  number  of  targets  is  large. 

•  There  is  no  provision  for  other  than  straight-line  tracks.  No  change  of  target 
course  is  permitted. 

•  The  function  keys  must  be  pressed  and  released  quickly  to  avoid  input  errors. 
Practice  sessions  can  serve,  in  part,  to  familiarize  the  user  with  proper  striking  of  the 
keys. 

•  The  program  requires  the  FP-51  accessory  pack  in  order  to  execute  selected 
graphical  and  mathematical  commands. 


AN  EXPERIMENT  TO  DEMONSTRATE  FEASIBILITY 

To  demonstrate  the  feasibility  of  the  air  defense  game  as  a  research  vehicle, 
volunteers  were  recruited  to  participate  in  a  preliminary  experiment.  The  experiment 
was  exploratory  and  was  designed  to: 


•  Illustrate  possible  research  uses  of  the  air  defense  game. 

•  Checkout  software  and  procedures  in  an  on-line  environment. 

•  Test  the  feasibility  of  extracting  detailed  measures  of  human  performance. 

•  Ascertain  the  amount  and  type  of  training  required  to  reach  stable  performance 
at  moderate  levels  of  difficulty. 

•  Derive  indices  of  performance  that  are  properly  sensitive  to  variations  in  task 

load. 

Subjects 

The  subjects  were  17  Navy  enlisted  men  (E-4  to  E-6)  waiting  to  enroll  in  advanced 
technical  courses  at  the  Naval  Training  Center,  San  Diego.  Their  median  age  was  19 
years;  the  range  was  from  17  to  28.  None  of  the  subjects  were  experienced  or  trained  in 
the  use  of  computers.  Participation  was  voluntary  and  informed  consent  was  solicited  in 
accordance  with  SECNAVINST  3900.39A. 

Procedure 


Two  subjects  were  run  at  a  time,  each  operating  a  separate  computer.  Neither 
subject  could  see  the  other's  keyboard  or  display.  The  testing  room  was  dimly  illuminated 
in  order  to  minimize  glare  on  the  display  screens. 
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The  subjects  were  first  given  a  briefing  that  described  the  rationale  for  the 
experiment  and  gave  an  overview  of  the  air  defense  game.  This  was  followed  by  a 
detailed  introduction  to  the  game,  the  activities  required  during  an  engagement,  and  the 
type  of  feedback  to  the  player.  Participants  next  played  two  or  three  practice 
engagements  to  become  familiar  with  the  display  and  the  data-entry  keys.  There  were  six 
targets,  two  at  each  speed,  in  each  of  the  practice  runs.  The  experimenter  instructed  the 
subjects  in  interpreting  the  display  and  in  the  proper  use  of  the  function  keys.  The 
experimenter  emphasized  the  importance  of  avoiding  hits  on  the  ship  and  of  kil'ing  targets 
at  the  greatest  possible  range,  but  no  explicit  rules  were  given  as  to  when  to  fire.  Rather, 
these  were  to  be  learned  during  the  course  of  play. 

After  the  briefing,  practice,  and  instruction,  the  subjects  began  the  training  sequence 
proper.  They  were  required  to  master  each  of  four  blocks  of  progressively  more  difficult 
types  of  engagements.  Training  was  self-paced,  and  a  skill  rating  of  Rj>75  on  three 
consecutive  engagements  was  the  criterion  for  proficiency  and  the  prerequisite  for 
advancing  to  the  next  block.  Two  training  sequences  were  used  and  subjects  were 
assigned  randomly  to  one  or  the  other.  The  two  sequences,  denoted  as  A  and  B  in  Table  2, 
are  identical  in  blocks  1  and  4.  They  both  progress  from  an  easy  task  (18  targets  at  low 
tempo)  to  a  moderately  difficult  one  (36  targets  at  high  tempo). 


Table  2 


Number  of  Targets  and  Tempo  of  Operation  in  the  Two  Training  Sequences 


Training 

Sequence 


_ _ Number  of  Targets-Tempo  of  Operation _ 

Block  1  Block  2  Block  3  Block  4 


A  18-low  18-intermediate  36-intermediate  36-higl 

B  18-low  36-low  18-high  36-higl 


Tempo  was  deemed  to  be  more  important  than  the  number  of  targets  in  determining 
task  difficulty.  With  this  in  mind,  note  that  sequence  B  contains  an  abrupt  shift  in  tempo, 
between  blocks  2  and  3,  and  that  sequence  A  provides  two  blocks  of  training  in  the 
intermediate  tempo.  It  was  hypothesized  that  subjects  using  sequence  A  would  complete 
training  more  quickly. 

After  a  subject  reached  criterion  in  block  4,  ten  additional  engagements  were  run  in 
the  36  target-high  tempo  condition.  These  were  identical  to  those  of  block  4,  except 
that  there  was  a  programmed  30  second  rest  period  between  engagements.  At  the  end  of 
the  fifth  engagement,  an  auditory  monitoring  task  was  introduced.  The  subject  received 
instructions  and  six  minutes  of  practice  on  the  monitoring  problem  alone.  Then,  in 
engagements  6-10,  both  tasks  were  performed  concurrently.  The  monitoring  task 
continued  during  the  30-second  breaks  from  the  air  defense  game. 


iMMfltokl  L  -- 


I  ~I  rtUlMtetilf  i  IT  if1  > 
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The  monitoring  task  resembled  a  typical  communications  problem  that  might 
accompany  AAW  operations  and  was  selected  because  of  its  minimal  sensorimotor  conflict 
with  the  air  defense  game.  Specifically,  the  subject  listened  to  a  series  of  words  and 
numbers  and  wrote  down  the  digits  that  followed  each  occurrence  of  a  preassigned  word. 
Each  message  in  the  series  consisted  of  a  category  (speed,  altitude,  etc.)  followed  by  a  1- 
word  call  sign  and  a  3-digit  random  number.  To  illustrate,  suppose  the  subject  had  been 
assigned  the  call  sign  YANKEE  and  had  been  presented  the  auditory  input  stream, 
"ALTITUDE  BRAVO  EIGHT-ZERO-SIX.  .  .SPEED  YANKEE  TWO-FOUR-THREE.  .  .  ." 
The  subject's  task  was  to  write  "243"  on  a  response  sheet.  For  this  subject,  all  messages 
with  a  call  sign  other  than  YANKEE  were  to  be  ignored. 

The  series  of  messages  was  recorded  on  tape  and  played  through  Audiotronics  HS14 
headphones  at  a  rate  of  one  message  per  3.6  seconds.  Thus,  100  messages  would  be 
presented  in  6  minutes,  which  was  the  approximate  duration  of  an  engagement,  including 
the  rest  period.  Each  of  10  different  call  signs  appeared  at  10  random  times  in  every 
series.  A  given  subject  had  the  same  call  sign  throughout  all  sessions. 

Testing  was  scheduled  for  3  hours  on  each  of  2  consecutive  days.  At  the  end  of 
testing,  the  subjects  were  interviewed  for  their  evaluation  of  the  task,  and  their  questions 
about  the  experiment  were  answered. 

Results 


The  feasibility  of  the  software  concept  and  its  implementation  were  confirmed.  The 
on-line  collection  of  performance  data  proved  feasible  and  straightforward,  and  there  was 
no  disruption  to  the  user.  Minor  problems  were  found  in  the  interrupt  routine  that 
services  the  function  key  inputs,  and  these  were  corrected  in  the  final  version  of  the 
program. 

Due  to  an  error  in  programming  design,  the  inter-update  interval  in  the  experiment 
was  allowed  to  vary  from  7.0  seconds  in  the  18  target-low  tempo  condition  to  9.8  seconds 
in  the  36  target-high  tempo  condition.  The  resulting  duration  for  these  engagements 
ranged  from  4.2  to  4.9  minutes.  The  effect  of  this  variation  was  to  give  the  subjects  more 
time  per  update  in  the  more  difficult  engagements.  While  this  error  did  not  affect  our 
interpretation  of  the  results  that  follow,  it  constituted  an  undesirable  and  unintentional 
manipulation.  The  program  was  modified  to  equate  the  intervals  at  1 1.7  seconds  for  all  12 
combinations  of  the  tempo  and  number  of  targets.  This  modified  version  of  the  program 
appears  in  Appendix  A  and  is  the  one  recommended  to  potential  users. 

Skill  Acquisition 

Only  8  of  the  original  17  subjects  completed  the  training  regimen.  Eight  others  had 
conflicting  duty  assignments  and  one  failed,  after  30  training  engagements,  to  reach  the 
performance  criterion  for  Block  1;  data  for  these  subjects  have  been  excluded  from  the 
analysis  given  below.  Of  the  remaining  eight  subjects,  three  received  training  sequence  A 
and  five  received  sequence  B.  The  overall  mean  number  of  engagements  to  complete  the 
four  blocks  of  training  was  39.1  (median  =  40;  range  =  19  to  73).  The  mean  figure  is 
equivalent  to  2.9  hours  of  playing  time. 

The  number  of  engagements  required  to  complete  the  A  and  B  sequences  did  not 
differ  significantly  by  the  Wilcoxon  rank-sum  test;  the  medians  were  45  for  sequence  A 
and  35  for  sequence  B.  This  was  contrary  to  the  expectation  that  the  A  sequence  would 


provide  a  smoother  transition  in  difficulty  and  would  thereby  enhance  acquisition. 
Apparently,  the  rate  of  acquisition  was  insensitive  to  both  tempo  and  the  number  of 
targets.  The  Kruskal-Wallis  test  was  employed  to  examine  the  interaction  between  blocks 
and  training  sequence  (Bradley,  1968,  pp.  138-141).  The  interaction  was  nonsignificant, 
indicating  that  performance  for  the  four  component  blocks  did  not  reliably  differ  as  a 
function  of  the  training  sequence  manipulation. 

After  collapsing  the  nonsignificant  sequence  variable,  the  median  numbers  of 
engagements  to  criterion  were  9.0,  3.5,  5.5,  and  13.0  for  the  Blocks  1-4  respectively. 
These  differed  reliably  by  the  Friedman  test,  S(3)  =  153.5,  p  <  .01.  It  appears  that  once 
the  basic  skills  of  the  game  are  acquired  (in  the  easy  condition  of  Block  1),  then  these  are 
sufficient  to  prosecute  engagements  of  moderate  difficulty  (Blocks  2-3).  Additional  skills 
must  be  mastered,  however,  to  meet  the  increased  demands  of  the  36  targets-high  tempo 
condition  of  Block  4. 

Because  there  were  large  individual  differences  in  the  number  of  engagements 
required  to  reach  criterion,  performance  scores  were  Vincentized  (see  Hall,  1966,  p.  64) 
into  quartiles  to  further  examine  the  course  of  acquisition.  Figure  10  shows  the  mean 
skill  rating  (R)  by  quartile  in  each  of  the  four  training  blocks.  The  positive  trend  within 
each  block  makes  it  clear  that  performance  does  indeed  improve  over  the  four  quartiles. 
This  effect  was  significant  by  a  Friedman  test,  xZ(3)  =  19.7,  p  <  .01.  Bradley's  (1968) 
suggestions  for  extending  the  Friedman  test  were  used  to  assess  possible  interactions; 
quartiles  did  not  significantly  interact  with  either  block,  x2(12)=8.33,  or  with  the 
training  sequence  (A  vs.  B),  x2(4)  =  2.0.  Thus,  the  course  of  acquisition  was  similar  within 
a  training  sequence  and  regardless  of  the  sequence  type. 

In  summary,  about  3  hours  with  either  training  regimen  produced  proficient  per¬ 
formance  in  quite  difficult  air  defense  problems.  The  course  of  skill  acquisition  was 
insensitive  to  the  variations  in  the  sequence  of  engagement  types.  Basic  skills  were 
acquired  largely  through  early  practice  and  transferred  readily  to  moderately  difficult 
engagements.  Significantly  more  practice  was  required  to  establish  a  high  level  of 
proficiency,  though  well  short  of  mastery,  in  the  more  demanding  air  defense  problems. 

Dual-task  Performance 


Only  five  subjects  were  available  to  complete  testing  in  the  dual-task  sessions.  The 
skill  ratings  declined  significantly  when  the  auditory  monitoring  task  was  added  to  the  air 
defense  problem.  Presumably,  the  information  processing  demands  of  the  two  concurrent 
tasks  exceeded  the  subjects'  resources.  The  differences  between  single-  and  dual-task 
performance  are  summarized  in  Table  3. 

Analysis  of  other  air  defense  game  data  corroborated  the  decrease  in  the  skill  rating 
observed  in  the  dual-task  condition.  The  range  in  miles  at  target  intercept  decreased 
while  the  number  of  hits  on  the  ship,  the  number  of  inflight  launches,  and  the  number  of 
splashes  increased. 

A  decline  in  performance  was  also  evident  in  the  auditory  monitoring  task.  When  it 
was  the  sole  task,  performance  was  virtually  perfect.  When  it  was  performed  concur¬ 
rently  with  the  air  defense  game,  detection  of  the  call  sign  messages  exhibited  a  small  but 
significant  decline.  Most  errors  (92.3%)  were  due  to  omissions.  From  this  pattern  of 
results,  it  is  clear  that  the  subjects  did  not  perform  one  task  strictly  at  the  expense  of  the 
other;  rather,  performance  in  both  tasks  was  significantly  affected. 


BLOCK  1  BLOCK  2  BLOCK  3  BLOCK  4 


1234  1234  1234  1234 

Quartile  Quartile  Quartile  Quartile 


Note.  Data  from  training  sequences  A  and  B  were  combined. 

Figure  10.  Vincentized  mean  skill  ratings  (R)  by  quartile  for  the  four  training  blocks. 


Table  3 

Comparison  of  Performance  in  Single-  and  Dual-Task  Conditions 


Performance 

Characteristic 

Median  Scores 

Single-task  Dual-task 

Condition  Condition 

Pa 

Skill  rating  (R) 

77.0 

68.0 

<  .01 

Target  intercept  (kill)  range 

15.8 

15.0 

<  .01 

Hits  on  ship 

0.0 

2.0 

<  .01 

Inflight  launches 

3.5 

6.7 

<  .025 

Premature  launches  (splashes) 

6.0 

5.0 

>  .10 

Monitoring  task  errors*5 

0.0 

1.0 

<  .01 

Wilcoxon  signed-rank  test. 

^The  range  of  errors  was  0  to  1  when  the  monitoring  task  was  performed  by  itself;  the 
range  was  0-5  in  the  dual  task  condition. 
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The  relation  of  task  load  (single  vs.  dual)  and  target  speed  (fast,  medium,  or  slow)  to 
the  observed  firing  lag,  L-L*,  is  shown  in  Figure  11.  The  auditory  monitoring  task 
significantly  increased  firing  lags,  W  (5)  =  0,  p  <  .05,  Wilcoxon  signed-rank  test.  The  main 
effect  of  target  speed  was  also  significant,  S(3,5)  =  38,  p  <  .05  (Friedman  test).  Firing 
lags  were  longer  for  slow  targets  than  for  either  fast  or  medium  ones.  We  cite,  as  the 
most  likely  reason,  the  low  priority  that  the  rules  impose  on  slow  targets.  Since  they 
move  a  shorter  distance  between  updates,  smaller  penalties  are  incurred  (in  terms  of 
killing  range)  by  firing  at  fast  and  medium  speed  targets. 

The  interaction  between  target  speed  and  task  load  was  not  significant,  suggesting 
that  the  demands  of  the  monitoring  task  induced  a  general  decrease  in  attention  to  the  air 
defense  game. 


Note.  A  lag  of  2  means  that  the  player  fired  on  the  second  update  after  the  target 
entered  firing  range. 


Figure  11.  Mean  firing  lag  as  a  function  of  target  speed 
for  single-  and  dual-task  conditions. 
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Evaluation  by  Users 

Unlike  most  laboratory  tasks,  the  air  defense  game  was  fun  and  the  subjects  were 
able  to  maintain  their  interest  in  it  for  periods  of  3  to  4  hours  at  a  sitting.  They  reported 
that  the  feedback  displayed  at  the  end  of  each  engagement  was  especially  helpful  in 
diagnosing  their  tactical  weaknesses  and  in  providing  objective  performance  standards 
with  which  to  monitor  their  progress.  The  capability  for  presenting  engagements  of 
varied  difficulty  was  also  regarded  as  a  desirable  feature. 


CONCLUSIONS 

This  effort  demonstrates  that  the  air  defense  game  is  an  effective  vehicle  for 
investigating  human  performance  processes  in  AAW  threat  analysis.  The  game  requires 
sustained  attention  to  a  complex  and  interactive  "hostile"  environment,  provides  proper 
experimental  control  of  relevant  variables,  and  affords  detailed  quantitative  measure¬ 
ments  of  human  performance  that  can  be  compared  to  that  of  a  mathematically  ideal 
information  processor.  The  hardware/software  system  provides  flexibility  and  portability 
in  an  inherently  motivating  task  that  taps  the  cognitive  skills  required  in  selected 
command  and  control  environments. 

Proficient  performance  can  be  attained  with  moderate  amounts  of  practice.  Varia¬ 
tions  in  the  number  of  targets,  target  speed,  and  the  pace  of  operations  produce  reliable 
effects  on  performance.  The  task  is  sensitive  to  the  workload  demands  of  a  concurrent 
auditory  monitoring  task. 


RECOMMENDATIONS 

1.  The  air  defense  game  should  be  exploited  as  a  tool  in  future  research  on  the 
threat  analysis  problem. 

2.  The  air  defense  game  should  be  used  to  determine  limitations  in  human 
information  processing  and  how  strategies  for  threat  analysis  change  as  a  function  of  task 
load. 


3.  Performance  in  the  air  defense  simulation  should  be  used  as  a  dependent  measure 
for  other  military  research  applications  (e.g.,  the  effect  of  extended  effort  or  sleep  loss). 
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7  RUN  170 

3  REM - (unction  Key  servicing 

4  T»=T»4"I" 

6  COSUB  A  OF  2680 . 50 

7  RETURN 

8  T«cT«S-2' 

10  CCSUB  A  OF  2690.50 

11  RETURN 

)2  t*-t *43" 

14  C0SU8  A  OF  2690.50 

15  RETURN 

16  T*=T*&"<- 

18  CCSU8  A  OF  2690.50 

19  RETURN 

23 

21  C0SU8  A  OF  2690.50 

22  RETURN 

24  T*=:T*4"6" 

25  G0SU8  A  OF  2690.50 

27  RETURN 

28  T*=T*4"7“ 

50  C0SU8  A  OF  2690.50 

31  RETURN 

32  T  *  =  T  *4" 8 " 

34  GOSUB  A  OF  2690,50 

35  RETURN 

36  T*=T*4"9" 

38  COSUB  A  OF  2690,50 

39  RETURN 

40  T*=T*4"0" 

41  GOSUB  A  OF  2690.50 

42  RETURN 

49  REM - - - —  -  i  noppropr  I  ot e  Keypress 

50  T  *  = "  " 

60  RETURN 


08  REM 
09  REM 

100  PEM  AIR  DEFENSE  GAME 

I  10  REM 

).?C  REM  R  L  Hershmon.  F.  L.  Greitzer,  &  R  T.  Kelly 

130  REM  Command  ond  Support  Systems,  Code  302 

M2  rfm  Navy  Personnel  Reseorcn  4  Development  Center 

153  REM  Son  Diego.  CA  92152 

163  REM 

1  T0  pem - f  ind  next  doto  f  i  le 


180  PRINT  833.0  0.0. 1 

190  FI =5 

230  FIND  FI 

210  INPUT  833  AS 

220  ASsSEG I  AS , 9 ,  I  1 

230  IF  A • = " L "  THEN  260 

240  F 1 *F 1 ♦ 1 

250  GO  TO  200 

260  PRINT  §33.0.0,0.0 

270  SET  DEGREES 

280  SET  NOKEY 

290  A?2 

300  ZxRNOl-l > 

310  REM - print  Instructions 

320  PRINT  "UHHHHHHA1R  DEFENSE  GAME  INSTRUCTIONS  TOUR  SHIP  IS  U"; 

330  PRINT  ••NDER'ATTACK  BY  1NC0MINC  MISSILES  WITH  VARYING  SPEEDS  AND"; 
340  PRINT  “  LAUNCH  TIMES.  YOUR  TASK  IS  TO  M0NIT5R  THE  RADAR  DISPLAY". 
350  PRINT  "  AND  DESTROY  THEM.  THE  PRIORITIES  OF  THE  ENGAGEMENT  ARE.-  “ 
360  PRINT  "  111  AV010  BEING  HIT.  12/  AVOIO  SPLASHING  YOUR  OWN  "; 

370  PRINT  "MISSILES  BY  LAUNCHING  TOO  EARuY...Y0uR  WEAPONS  *; 

380  PRINT  "RANGE  IS  20  MILES,  WHICH  *  IS  THE  INNER  CIRCLE  OF  THE"; 

390  PRINT  "  RADAR  DISPLAY  13)  DESTROY  INCOMING  MISSILES  AS  SOON  "; 

400  PRINT  "AS  POSSIBLE  '*  AFTER  THEY  ENTER  YOUR  WEAPONS  RANGE.  " 

410  PRINT  "  (41  AVOID  LAUNCHING  A  MISSILE  IF  YOU  ALREADY  HAVE  ONE" 

420  PRINT  "  IN  FLIGHT  ON  THE  SAME  TARGET.  TOUR  SKILL  RATI"; 

430  PRINT  "NG  10-100)  WILL  INCLUDE  A  12-POINT  PENALTY  FOR  EACH  HIT  "; 
440  PRINT  "SUSTAINED  ANO  A  2-POINT  PENALTY  FOR  EACH  INFLIGHT  LAUNCH.  * ; 
450  PRINT  "  THE  MAXIMUM  KILL  RANGE  JS  20  MILES'  FIVE  POINTS  ARE  OEDUC"; 
<60  PRINT  "TEO  FOR  EACH  MILE  THAT  YOUR  AVERAGE  KILL  RANCE  IS  UNDER  20." 
470  PRINT  "i  TO  LAUNCH  A  MISSILE,  USE  THE  TEN  WHITE  KETS  AT  THE"; 
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480 
493 
:.eo 
51  e 
sro 

5  50 
540 
553 
560 

5  70 
582 
500 
603 

6  10 
S3C 
6J0 
54C 
650 
660 
670 
680 
690 
700 
7  1  0 
7  20 
730 
740 
750 
760 
770 


PRINT  "  UPPER  LEFT  OF  THE  A £ YBOARC  ENTER  ThE  TeO-DIGJl 

PRINT  "TRACA  NUMBER  OF  ThE  TARGET.  A  READOUT  WILL  THEN  INFORM  " 

PRINT  YOU  OF  A  SUCCESSFUL  LAuNCH--OR  AN  ERROR 

PRINT  -AN  ERROR  OCCURS  IF  YOU  AE*  A  NUMBER  INCORRECTLY". 

PRINT  "  OR  IF  YOU  LAUNCH  A  MISSILE  UNNECESSARILY  " 

PRINT  -j  GOOD  LUCA  the  Fate  of  your  ship  lies  in  your  HANDS'" 


PRINT  -j  Press  RETURN  to  coni -nue 

INPUT  as 

REM- - cleanup  memory  ona  ge  >  ployer  3  D. 


PRINT  'LJJENTER  YOUR  NAME 
INPUT  Si" 

21  --0 
T  7-54 
Its-  "L»«" 

US:  £«■ “ 

DELETE  ?3.S$.F3.C3.A3.L3.X0.A5. TS. A6. A7. A8. AS.H9.LS. At . A5. A4. 9, 7? 
SET  NONE Y 
A  =  2 
IS-." 

REM - seleci  lenpc  ond  torget  density 

WSa'LOW 

.ys="Intermeoiate 

YSi'HlCH 
OS= ' Proc 1 i ce" 

PRINT  "JJSELECT  TEMPO.  I=".WS."  2='.X S.  3=".YS 

PRINT  “*  4-" . OS.  ■ 

INPUT  0 

IF  0<l  OR  D>4  THEN  740 
IF  D<>4  then  8ie 
N  0  =  2 


780  A  =  2 
700  w 7 r 44 3 
800  GO  TO  890 

810  PRINT  " JJJSELECT  TARGET  DENSITY  II.  2,  3.  OR  4 ) ;  " . 

820  INPUT  N0 

830  IF  N0< 1  OR  N0>4  THEN  820 
840  N0=N0»6 
850  A=N0/O 

863  REM - - set  97  counien  10  eauote  update  lines 

8^0  AS 5 SEC  I "083095  I  000250220 180  I  000600200400 1000". 3»N0/2-3*D- II  .  3 ) 

8 83  W7=YAL  (AS  I 

800  N=3«N0 

900  W8=N=72  and  0=3 

912  PRINT  "JJJJVe'll  Gegin  ,n  a  f e*  seconds 
920  DIM  A5inI?R! IN) 

930  REM - random  .  za  i  i  on  for  entry  times 

94 C  CALL  "FLASH- 
950  R’=INT (RNOl 11-0  5) 

960  FGR  1=1  TO  A 

9^0  00  TO  0  OF  1040.1010,980.1040 

960  A5  I  I *2»AI =5* I -Rl  l  I -2*A  I 

930  A5  I-2YA-N01  =  4*  3  -3-RI  T  1  -2*A-N0> 

1000  *5 ( I -2SA-2SN0 I = I -R I U-2»A-2sN0> 

1012  A5 I  I -A  I =5sl -4-RI  I  I -A! 

1020  AS  ;  I-A-N0) =4* I* l-R 1  I I-A-N0: 

1030  A 5 l I-A-2YN0I  =  1*RI I  I -A -2SN0 I 
I  040  A« i; 1 =5»I«RI (I) 

IC50  A5  :  I-N0)  =  4*I-I*R1 I I-N0) 

1060  A5I I-2SN01 =2*l-t-R1 < I-2SN0! 

■07C  NEXT  I 

1080  REM - assign  trocx  nos.  speeds,  ongles,  and  h.t  times 

i e30  Din  P3!90i , S3 (90i . h9(n; . ls  ;  1 2001 , a  1  1901 , as  001 ,*91901 . xg ig0i , w ini 
I  100  S3=0 

I  1  10  CALL  "MFLASH" 

I  120  A  1 =FNX I  4  I 
I  I  30  A8  =  FNX(I 1 -g 
t I  40  V  =  FNX ( I  1 
I  150  REM 

I  160  FOR  A  =  I  TO  g0 
1170  B  =  A 1  (A  I 
1 100  C  =  A8  T  A  I 
I  I  90  CALL  "MFLASH" 

i20e  j=intirnoii i»a0i-i 
1210  J1  =  INT(RN0(1 1 ?90) -  1 
I22e  REM 

I23e  A  1  1A)=AI tj) 


A~2 


m 


n 


1240  A  I (J>  = 8 
1250  A8(M=A8(J1  I 
I2G0  A B  t  J 1 >  =  C 
1270  NEXT  A 

1280  OIM  Al (Nt .A8<N) .A4IN1 
1200  F 08  1*'  TO  N 
I  520  A9I I  I =  A I  I  11/4 
1510  IF  ]<  =  N0  THEN  1390 
1720  IF  1<=2»N0  THEN  1360 
1350  A4 I  I  1 =28*A5 I  I 1 
1 540  S3  I A9( I  1 1  =  I  . 6 
1350  GO  TO  I  4  1  e 
t  360  A 4 ( I  I =I6«A5I I 1 
1  570  S3IA9II I  1-3 
1380  GO  TO  1410 
1  390  *4  I  I  1  =  1B«A5t I  1 
1400  S3  I A9 l  I  1  1  =5 
1410  NEXT  I 

1420  R£M - draw  circles 

I <30  PACE 

1440  CIIT  X  1 1  011  .  r  (101)  ,T2(N) 

1450  23*46 
1460  C0Su8  1500 
1470  20=20 
1483  GCSUB  1500 
1490  GO  TO  1570 
1503  CALL  "MFLASH" 

1510  X=SIN(FNX(3  61)*Z0-77 
1520  Y=C0S(FNX(3  61 !*Z0»50 
1530  XIIU-XIII 
1540  REM 

1550  CALL  "OFLASH".X.Y 
RETURN 

1570  MOVE  T7-0  5.48.5 

1580  PRINT  832 ,185 

1590  PRINT  ■•♦t  "  ;  fIS  ;  "  J  ";N« 

1600  OELETE  X?OT 

1610  RET1 - initialize 

1623  DIM  F3I90! .G3I90I .531 1801 .L3I 180) .TSI201 .A6IN) .A7IN1 .A5IN1 

1630  0=1 

1640  M=C 

1650  a9=C 

1663  17  =  0 

1 6?C  L8  =  C 

1680  l9  =  0 

1693  A6=0 

1703  A  7  =  0 

1713  P3  =  44 

1  723  FOR  1  =  1  TO  N3 

1  730  P3 1 A9 I  I  1 1 =45 

I  743  NEXT  I 

I  750  H0  =  0 

i 7ee  xg=e 

1773  H9=l 
1780  L 1 = 1 
1793  SET  HEY 
I80C  A  = 1 

1810  REM - start  new  eoocM 

1823  M=0 
i83e  h=i 

1840  CALL  "FLASH" 

1850  A6=ABStAS!*H9 
I860  H9=l 

1870  REM - determine  target  stotus 

1880  FOR  10=1  TO  N 

1890  GC  TO  2»IA5II0)>U)*W8  OF  1900,2330.1980 

1930  IF  U  =  A4 I  101  THEN  3420 

1 9 ! 0  X9IA9I 1011=1 

1920  IF  P3IA9II01 l<44  THEN  I960 

1930  A4  =  M»I 

1940  H5IM)  =  I0 

1950  GO  TO  1980 

I960  CO  TO  A6 1 101  +  1  OF  1980.2370 
1970  A61I01=t 
1980  NEXT  10 

1990  REM - »ol>e  rodor  eeeep 
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2300  CALL  -nFLASH- 

?0I3  F3  = f 2*X9- I ) * ISINIFNX I  4 ) >»P$*T7) 

2020  C3=C0S IFNX I  4 11 »P3*50 
2353  HCn 

2343  FUR  1*1  10  90 
2302  CA.L  'NF_ASH" 

2063  I  =  2*  I  -  I 
20  73  K.31JI  I-F5I1) 

2283  F5IJMUF3HI 
2 292  L  3  :  J  l  : =C3  1 1 ) 

2102  l3Ul*l »  =C3 l 11 
2  I  12  PEN 
2'2C  NEXT  r 
.?  >  52  CAlL  "XFlASH" 

2  I  43  A3--  -  A8S  IASI 
2:5C  H=-l 

2 1 60  CALL  ’ OFLASH  '  .  a3 , L  3 
2 ' 10  !F  a 4  =  0  THEN  2200 

2  103  REM - (display  lorge-  nos 

2:90  FOR  J2=!  70  A4 
220C  C‘.L  ""FLASH" 

2210  X  i  sS  1 N  ( A  1  1  A 5  I J2 1  1  1148  5*  T  7-»  0 . 5 

2220  y7-cCS!A1 (A5IJ21 1 1*48  5‘49 

2230  REFI 

2240  TOVE  X7  ,  V7 

2250  PRJN’  -h”.A8(A5IJ21  1  , 

2260  NEXT  J2~ 

22 70  REN  -  - - corput  e  "ex t  pos  i  t  1  cn 

2280  call  flash- 
2290  P3=PS-X9*S3 
2  32  0  P3--0  MAX  P3 
2310  U-U->1 
2320  CO  TO  1820 
2330  FOP  W6=l  TO  W7 
2332  NEXT  wg 
2352  CO  to  t g80 

2362  PEri - plot  Player's  missile  track 

2370  A7l]0)*A7f 101*5 
2583  IF  A 7 1 |0i >23  THEN  2500 

2593  IF  P31A91 101 lxS3IA91 101 1-A7I 101<=0. 1  Th£N2550 

2-’C3  C‘LL  PFlASH- 

2 !  ! 0  X  I .SINIA)  1101  )*A7 ! J0) .17 

2420  X  '  -.COSIAUI01  1XA7M01-50 

2430  x2-s ! N I  A l  11011*1 A7 (101-11. T7 

2443  7  2  =  C0S< A  I  I  101 ) » IA7I J01 -  1 1 *50 

2450  PEP 

24  60  HO  vE  X  I  . T 1 

24  7C  ORA*  >2,17 

2483  CO  'C  1980 

249C  RE" - missile  sclosned 

2500  L9--IBM 
2513  ‘71 101 -0 
2520  ‘61101=0 
2530  CO  T0  1980 

2540  R£n - desi'Oyed  o  icrgei 

2550  PRINT  -C  . 

2560  "OVE  77750 
2570  ROTATE  90- A  1  I  10! 

2580  R0RA9  45.0 

2590  xg  i  a9  i  ten  ,0 

2603  K9=k9xP3(A9(  10)  )tS3IA9U0)  1 

2610  451 10) =  AS  I  101*2000 

2623  A61 101 =  0 

2630  MiHlil 

2640  IF  AI-.H0ON  THEN  1  980 
2S5E  !0=N 
2660  NEXT  |0 
2670  CO  TO  3560 

2680  R£n - - - - - assemble  targe)  no  for  leunch 

2690  GO  TO  LEN I T* M 1  OF  2700,3320.2730 
2 700  TSxSEGITS .1,21 
2710  CIN  X6. V6 
2720  GO  TO  3240 
2730  T0=VAL(TSI 
2740  CIN  X6.X6 
2750  REN 


lookup  target  * 


?7B3  T?=i0^*e 

.>  '  ?2  T2=T2*V 

2  ’83  J=SuMIT2l 

p 7 9c  |F  J=0  THEN  3210 

260a  Jf  *BSl*6UH  =  l  Then  3182 

2810  IF  \9IA9U1I=0  THEN  3210 

2820  CALL  ' MFLASh" 

2B3E  X3--S1NIAI  C  J  1  1  »5-»  T  7 
2P40  Y3=20S l A  I  I J I  1 *5»50 
.-053  M-.5IMAIIJII«'TT 
2R30  Y4rC0S(AI  I J I  1*4-50 
2072  Pin 

2833  REM - launch  the  *issile 

?E 90  MOVE  X3.Y3 

2930  DRAW  X4.Y4 

2910  A6!JI--I 

2920  H9 ( J 1 =H 

2930  A?; j: =5 

2940  LS-L9-I 

2950  I*-STR(600-A8(J1 ) 

2960  GOSUB  3340 

2970  REH - good  launch  i  nf  cnn>  player 

2980  FCR  S  = 1  TO  6 

2990  TOVE  I . 792-Z I *5 . 376. 86 . 9! 2 

30ee  PRIM'  T * ; 

3010  FOR  SI =1  TO  8 
3020  NEXT  SI 
3030  NEXT  S 

304e  IF  P3IA9IJ1 I-S3IA9IJ) 1>5  THEN  3300 

3050  REM - destroyed  target 

3060  PRINT  -C“ ; 

307e  MOVE  T7.50 
3080  ROTATE  90-AI I J 1 
3C9e  RCRAW  45.0 
3100  X9IA9IJ) 1=0 
31  ie  AS ( J  >  =0 

3l2e  N9  =  N0  +  0  MAX  P3IA9IJ1  1-S3IA9U)  ) 

3  I 3e  A5(J)=A5lJ)-2000 

3140  K  1  sK I » 1 

3153  IF  M«h0<>N  THEN  3300 
3160  GO  TO  3560 

i)72  REM - error ; i n-Y 1 tght  lounch 

3183  l 7 - L  7- I 

3190  1*=$TR(700-A81J) I 

3200  GOSUB  3340 

3210  REM . . error  i  nf  orn  ployer 

3223  GCSU8  3240 

3233  RETURN 

3240  FCR  S=l  TO  6 

3253  MOVE  1.792-21*5.376.78  464 

3260  PRINT  T*. 

3270  FOR  S  I  =•  1  TO  8 
3280  NEXT  SI 
3290  NEXT  S 
3300  MOVE  X6.V6 
3310  T *  =  ** ” 

3320  RETURN 

3330  REM  load  L*  »itM  louncrt  Information 
3340  HSsSTP l U-99  5-H/2) 

3350  H*=SEG (HS .2,3) 

3360  I*=SECII*,2,  31 
3370  H*=H*4I» 

3380  L*=REPIH*.U  ,01 
3390  L I =L 1 *6 
34e0  RETURN 

3410  REM - torget  hit  player'*  ship 

3420  H0sHB ♦ I 
3430  A5  I  1 0 >  =  A5  I  10)  *  1000 
3440  X9IA91 10)  I  =0 
3450  PRINT  "CGG" ; 

3460  CALL  "MFLASH" 

3470  X8=SINIAI 110) )*48.5-T7+0.5 

34ee  Y8=C0StAM  10)1*48. 5-49 

3450  REM 
35ee  MOVE  X0 .  Y8 
3510  PRINT 
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j~re  ir  x.i*h0<>n  then  iqsb 

3530  I3--N 
3541)  NEXT  10 

3553  PEN - end  of  gome 

35.60  rOVE  0.4 
35’0  PRINT  "End  of  game 
3*-80  A-.? 

3590  F CP  1  = !  TO  700 

S6?c  ;--j 
36  10  NE*T  1 
3F.30  SET  NCfEf 

36  30  RE*1 - - sumnor  (  :p  ^erf  :rircrce 

36  4  C  z:n  CM3>  .C2fJ!  .  v*  15) 

3650  C--D 
3660  C.’-O 
3670  F  CP  x ’0  3 
3660  fcp  :  =  1  x  -  I  >»N0*  I  TC  x?N0 
3690  !F  l ; I <0000  ’HEN  3720 
3  '00  C?:x  i  =  C2  :M  ♦  ‘ 

S  ’  0  GO  to  3730 

3  720  0!  IM  --C!  HM 

3  ’50  NE«T  I 

? 740  NEXT  A 

3750  !F  X ’ -0  Then  377e 

3  760  x.g-xs/xi 

3770  print  "l  IhhmhhhaIR  defense  Sumary 
3780  CO  TO  D'CF‘3790. 38)0. 3830. 3640 
3790  QtxW* 

3803  CO  TO  3840 
3610  C»  =  x* 

3800  CO  TC  3840 
3830  Gt  =  Y* 

3840  PRINT  832.  18.0 

3850  PRINT  TETPO .  ".0*."  SPEED 

3660  Vtx’FAST'3 

3870  PRINT  USING  3920.V9.C2M  1  .Cl  I  1  > 

3880  V*= "MEDIUM” 

3890  PRINT  USING  3920 , V« . C2 12 1 . C 1  12 1 
39P0  V*  =  "SLOW 

39’0  PRINT  USING  3920  V* . C2  I  3 ) . C I  I  3 1 
39.’>0  MACE  8A.i0O.l6D 
3930  PRINT  -roTAL  X  I  L  L  S  *  ’  .  SUM  I C2 )  '  TOTAL  HITS  x  “  ;  Sufi  1C  1  > 

3640  PRINT  USING  JAVEfiACE  RANGE  FOR  XILLS  *  "20. 2D"  xs 

3950  PRINT  ' x I  I ; n A x  "x  231” 

3660  P*=  _NC”6f  MISSIlES  " 

i?'Z  PRINT  R«,  LAUNCHED  X  l9*L 7. Rt. "SPLASHED  *  ,L8 

3980  PRINT  -NO  OF  INFlICHT  L  a  UNC  HE  5  =  " . L  7 
3990  Px  IN’  MC0AX9/20-  I2»SUH!C  !  )  -2*L7) 

4030  PRINT  CJSXILL  RATING  X  , R  “AX  0."  I  PAX  =  1001" 

40IO  if  a*x  practice’  then  *\zz 

4C20  REN - s  1  ore  oo i o  on  t  cpe 

4030  PRINT  "Ooio  is  oemg  s'orea  on  i  ope 
4 C 40  x=LEML*) 

4053  FIND  F ' 

4360  haRx  l.30«N*300*l 
40 70  FIND  Fi 
4083  U  =  u*0  5-H/2 

4090  WRITE  933  S*. D.N. A  1  . A5, A8.L .L*.G1 ,C2 ,X3. l9, L8. L7.R, U 
4)00  CxOSE 
4113  F I =F l ♦ I 

4)23  PRINT  "JCO'TOU  WANT  TO  PLAT  AGAIN  IY/N1’ 

4130  INPUT  A* 

4140  IF  A*x"Y”  THEN  4 ) 80 
4 ' 50  IF  A$<>  N  '  THEN  4  I  30 

4  I  60  PRINT  "JTHANX  YOU  ’ 

4  I  70  END 

4  180  REM - Se>  disploy  poromeiers  for  nexi  gome 

4190  PAGE 
4200  A x 2 
4210  T  tx  " 

4220  SET  NOxEY 
4230  M*x"  -sn* 

4240  N»x"  "4N« 

4250  21=21*1 

4260  T 7  =  T 7 *5  376 

4270  GO  TO  121x51*1  OF  630.590 


’  .SA.  '•  NO  OF  ’ARGE’S  X  "  .  N 


*  x ills  *  hits 
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APPENDIX  B 

AIR  DEFENSE  GAME:  PROGRAM  VARIABLES 


VARIABLE 

NAME 

DEFINITION 

TYPE 

A 

Switch  for  servicing  function  keys 

Simple 

A$ 

Tape  file  header 

String 

A1 

Target  bearing(degrees) 

Array (N) 

A4 

Update  at  which  target  will  hit  own  ship 

Array (N) 

A5 

Target  entry  time:  plus  1000(hit)  or  2000 
(killed) 

Array (N) 

A6 

Own  missile  status  (l“ln  flight) 

Array (N) 

A7 

Own  missile  position 

Array (N) 

A8 

Target  track  number (TN) 

Array (N) 

A9 

Code  for  target  bearing*Al/4 

Array (N) 

B 

Temporary  storage:  bearing  randomization 

Simple 

C 

Temporary  storage:  TN  randomization 

Simple 

Cl 

No.  of  hits  (fast,  medium,  slow) 

Array (3) 

C2 

No.  of  kills  (fast,  medium,  slow) 

Array (3) 

D 

Tempo  of  operations 

Simple 

FI 

Magtape  data  file  no. 

Simple 

F3 

X-coordinate  for  target  display 

<i.cray(9U) 

G3 

Y-coordinate  for  target  display 

Array (90) 

H 

Missile  status 

Simple 

H$ 

Launch  data:  6  digits 

String 

HO 

No.  of  hits  on  own  ship 

Simple 

H9 

Missile  status 

Array (N) 

I 

Index  in  for-next  loop 

Simple 

1$ 

Launch  information 

String 

TO 

Index  in  for-next  loop 

Simple 

J 

Index  in  bearing  randomization 

Simple 

Jl 

Index  in  TN  randomization 

Simple 

J2 

Index  in  for-next  loop 

Simple 

K 

Loop  control:  NO/D 

Simple 

K1 

No.  of  kills 

Simple 

VARIABLE 

NAME 

DEFINITION 

TYPE 

K3 

Radar  sweep  array  for  FP-51 

Array(180) 

K4 

No.  of  targets  entering  this  update 

Simple 

K5 

Pointer  to  targets  that  enter  this  update 

Array (N) 

K9 

Accumulator  for  killing  range 

Simple 

L 

Final  length  of  L$ 

Simple 

L$ 

Player's  launch  history 

String 

LI 

Current  length  of  L$ 

Simple 

L3 

Radar  sweep  array  for  FP-51 

Array (180) 

L7 

No.  of  inflight  launches 

Simple 

L8 

No.  of  splashes 

Simple 

L9 

No.  of  good  launches 

Simple 

M$ 

Display  of  launch  status 

String 

N 

Total  no.  of  targets 

Simple 

N$ 

Display  of  launch  status 

String 

NO 

No.  of  targets  of  each  speed 

Simple 

P3 

Target  range 

Array (N) 

Q$ 

Text 

String 

R 

Skill  rating 

Simp.'-? 

R$ 

Text  for  feedback 

String 

R1 

0’s  or  l's  for  entry  time  randomization 

Array (N) 

S 

Index  for  flashing  TN  display 

Simple 

s$ 

Player  I.D. 

String 

SI 

Index  for  flashing  TN  display 

Simple 

S3 

Target  speed 

Array (N) 

T$ 

Input  of  launch  data 

String 

TO 

2-digit  launch  data 

Simple 

T2 

Array  for  track  number  lookup 

Array (N) 

T7 

X-coordinate  for  own  ship 

Simple 

U 

Update  counter 

Simple 

v$ 

Text 

String 
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VARIABLE 

NAME 

DEFINITION 

TYPE 

W 

Track  number  lookup:  1,  2,...,N 

Array (N) 

w$ 

Text 

String 

W6 

Index  in  for-next  loop 

Simple 

W7 

Counter  to  equate  inter-update  intervals 

Simple 

W8 

Logical  switch  for  game  parameters 

I  =  high  tempo,  N  =72;  0  =  otherwise 

Simple 

X 

X-coordinate  for  display  of  circle 

Array (101 ) 

x$ 

Text 

String 

XI 

X-coordinate  for  missile  track:  origin 

Simple 

X2 

X-coordinate  for  missile  track:  terminus 

Simple 

X3 

X-coordinate  for  missile  track:  origin 

Simple 

X4 

X-coordinate  for  missile  track:  terminus 

Simple 

X7 

X-coordinate  to  display  track  number 

Simple 

X8 

X-coordinate  to  overwrite  track  number 

Simple 

X9 

Target  status  for  display: 

0  =  inactive;  1  =  active 

Array (90) 

Y 

Y-coordinate  for  display  of  circle 

Array (101) 

Y1 

Y-coordinate  for  missile  track:  origin 

Simple 

Y2 

Y-coordinate  for  missile  track:  terminus 

Simple 

Y3 

Y-coordinate  for  missile  track:  origin 

Simple 

Y4 

Y-coordinate  for  missile  track:  terminus 

Simple 

Y7 

Y-coordinate  to  display  track  number 

Simple 

Y8 

Y-coordinate  to  overwrite  track  number 

Simple 

Z 

Dummy  variable 

Simple 

ZO 

Radius  of  circle 

Simple 

Z1 

Counter  for  repositioning  display 

Simple 

B-3 
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m 
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APPENDIX  C 


DATA  ANALYSIS:  PROGRAM  LISTING 


C-0 


mm 


ili0  RE  r. 
i 10  rem 
120  BE" 

130  REH 
i 40  rem 
ISC  REM 
160  REM 
: 70  REn 
180  INIT 
190  PRINT  "CENTER  FILE  NO .  t 
230  INPUT  FS 
210  FIND  VAL  IF*) 

220  RE  AO  fiS3  S* . 0 . N 

230  Din  Al IN) .A51N) .A81N) .0IN.5) ,CI <3> .C2I3> .0199) .YI3.4I 
240  READ  e33  Al . A5.A8.L 
2 00  Din  L*  1 L ♦ I  I 

260  READ  833.L4.CI .C2.N9.L9.L8.L7.R1 .U 
270  REM  ..  ,  Qisploy  Do  I  o  Summery 

280  TS*"F ILE  «"&F* 

290  PRINT  "LJAJR  DEFENSE  CANE  SUMMARY  '  ,  Ti ,  ’*  “  .  S* 

300  C**SEG("LO0  INTERMEDIATEHIGH  “■  ”.  1 2* ID- 1 ) •» I  .  121 

310  PRINT  “UN*  ;  N :  “  “  ;  C* ;  "  SPEED  I Kj  LLS I H I  TSJ " 

320  FCR  1*1 "TO  3 

330  0**S£C( "FAST  MEOIUMSLO0  “  , 6* 1 1- I ) ♦ I  . 6> 

340  PRINT  US  INC  ••8AI3DI8D"  C*.C2U).Cl  II) 

3S0  NEXT  I 

360  PRINT  "JTOTAL  KILLS  *  "  ;SUMfC2)  ;  ”  TOTAL  HITS  =  "iSUDICD 

370  PRINT  USING  JJAVE.  DISTANCE  FOR  KILLS  *  “ "2D . 2D" : K9 

380  PRINT  ’  ^LAUNCHES'*  “;L9»L7;  "  SPLASHES  =  “.L8;  "  INFLHES  *  ";L7 
390  PRINT  "'R*  " ;  ft  1  MAX  0 
400  INPUT  0* 

4)0  DIM  PIU.6) ,F 13) ,F4 13) .F5I31 ,FBI3) .F7ISI .HISUniCI I  MAX  1) 

420  READ  F.F4.F5.F6.F7 

430  DATA  2.5. 12.9. 15,28.5.3. I .6.45,44.44. I .2.4.5. 10.20 
440  N0*N/3 
4S0  0*0 
460  J*l 


470  R=0 
480  A 4-.U 
490  0*0 
500  Y-0 

510  REM . W*  1 oo  K-up  loble  for  torgets  by  trocA  no 

520  REn . 01  1  .  1  )*Entry  U,  0(1.2)  =  0p  1  i  mol  U  for  launch 

530  REn  0II.3>*U  of  player’s  Killing  launch 

540  REM  .  . 0(1.4)**  of  inflignt  lounches,  Q(I.5)*»  of  sploshes 

550  REn  . H*updotes  on  which  hills)  occurred 


560  FOR  11*1  TO  3 

570  FOR  I *N0» 1 1  I  -  1 )♦ I  TO  N0*I1 

583  WIA8U  )  )*i 

590  IF  A5!  I  X2000  THEN  620 

603  01 1 . I  I *A5 1 1 1 -2000 

610  CO  TO  650 

620  Oil. 1 1*A5I I )- 1000  ■ 

633  HtJl*0(J, !!*F4;it: 

640  J*J«I 

653  Of  1 . 2)*0( I , f ) +F  1 1 1 ) 

660  M«M  MIN  011,2) 

670  NEXT  1 
680  NEXT  II 


690  REn . Gei  lounch  info  from  L* i T I *Updoie .  T2*TrocK  No. 

700  REn . Find  sploshes  ond  ihflii#  lounches 


710  FOR  1*1  TO  L/6 
720  XS*SEG(L*. 6*1-5. 31 
7$0  T 1 *VAl ( X* ) -  1 00 
740  XS*SEG (LS , 6*1-2,31 
750  TjsVALIXS) 

750  IF  T2>699  THEN  870 
770  X*T2-600 

780  IF  Tt. >0(U(X) ,2)  THEN  850 

790  REn . A  splosh,  so  engaged 

800  OfV fXI ,5I.Of0tXl , 51+1 
810  FOR  J*T I  TO  T I «3 
820  RIJ.2)*RIJ,2)4) 

830  NEXT  J 
840  GO  70  890 
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C-l 


850  OIVtXI  . 3 »  *T 1 
860  CO  TO  890 
870  X-.T2-700 

880  0(WIXI.4)*QIVIX).41*I 
890  NEXT  I 

900  REN . Prim  doto  by  torget 

910  FOR  1-1  TO  3 
920  PRINT 

930  PRINT  -ANC  TN  E  LX  L  LAC  IN  SP 

940  FOR  J=N0X1I-I!*1  TO  I XN0 

950  O6--0U.  51-0U.2I  NAX  0 

963  x  1 1 . 1 ! ( I . I  I *0 ( J , 51 

9T0  vi ] .2! XY ( I .2) *06 

983  ti  : ,3!*vil .3) *06t2 

990  v  i  I  .  4) rY  ( | , 41 *0  I J . 4) 

1020  AS^SEG  ( "FITS'  ,  I .  I  1  • 

'010  Ct-SECf  'HK-.OIJ.31  niN  2.1) 

1020  PRINT  USING  1040.  J.AS.AI  (J)  ,A8(J)  .OU.  1  1  .0IJ.2)  .01J.3) 

1030  PRINT  US  INC  1050;O1j , 31-01 J ,21 , 0: J , 4) ,CIJ .5) ,C* 

1040  IHAGE  2C4X1A4X3D4X2D4X3D4XJ04X304XS 
1350  INACE  304X204X204X1 A 
1063  NEXT  J 
1070  INPUT  0* 

1080  NEXT  1 

1090  PRINT  -  IHHHHNEAN  LAC  SO  LAC  N  IN  SP“ 

I  100  FOR  1= I "T0~3~ 

1113  C4-SEG I" FAST  MED  SL09" .4*11-11*1,4) 

I  120  PRINT  C*; "  TARGETS 

1130  PRINT  USING  "2D  207XS" ,YI1 .21/C2II) 

1140  PRINT  USINC  "20.207XS"  SOR ! I Y 1 1 , 3) -Y  1 1 , 2 1 T2/C2 I  1 1 1 /C2 <1  I  1 
1150  PRINT  USINC  "207X207X20" iC2IIl.YII,41,YII,l) 

I  I  60  NEXT  | 

1170  X  =  YI 1 ,21«YI2.2I*Y(3.2) 

1180  S  =  Y ( I .31 *Y(2.31«Y(3. 3) 

1190  PRINT  USINC  ALL  TARGETS  " “20 • 2D7XS" , X/SUOI C2) 

1200  PRINT  USINC  "20T207XS" , SOR It S-XT2/SUN IC2) 1/SUNIC2) 1 
1210  PRINT  USINC  "207X2DS",SUntC21 . VII .41*712, 4)*Y(3. 4) 

1220  PRINT  USINC  "7X20" . Y < 1 . 11 *Y (2, 1 1 *Y ( 3. 1 1 

1230  INPUT  0* 


240  REN . Aanlys  1  s  Dy  update  .  R I J  .  1 1  =Ac  i  I  ve 

250  REN . R  ( J  .  21  ^Engaged  .  R I J . 3) *K 1 1 1  *  .  .  Rt J ,4)=HI is 

260  REM  . R  ( J  .  5 1  ^Unengaged .  R  I  J  .  6)  sM  i  seed  opp«  . 


1270  FOR  I  I  =  l  TO  3 

1280  FOR  i*N0X II 1-11*1  TO  N0X 1 1 

1290  K5=0 11.31-1 

1300  IF  OH.  31=0  THEN  1550 

1310  PEN  . Procees  the  kill 

I  320  X I sF6 1 1 1 1 -F5 1 1 1 1X10 II .31-01 1 ,  II) 

1330  X2=5 

1  340  KiQ  1 1 .31-1 

1350  IF  XI<=X2  THEN  1410 

I  360  X  I sX I -F5 1 1 1  1 

1370  X2xX2«5 

1380  K»K*I 

1390  CO  TO  1350 

1400  REN  . target  is  ociive 

14  10  FOR  J=0 1 1 .11  TO  K 
1420  RU,  1  )=R(J.  11*1 
1430  NEXT  J 
1 442  FOR  JsK*l  TO  U 
1450  RU,3)*R(J,31*I 
1460  NEXT  J 

1470  REN . torget  is  engaged 

1480  IF  K* 1 sO 1 1 , 3)  THEN  1520 
1490  FOR  JsQI I ,3)  TO  K 
1500  RIJ.2)*RIJ,2)*I 
1510  NEXT  J 

1520  IF  01 1,31*011,21  THEN  1690 
1530  CO  TO  1620 

1540  REN . Process  the  hit 

1550  FOR  J*0(  1 .  1 1  TO  0(1.  I1*F4(II)-1 

1560  Rl J. I) mRIJ. 1 1*1 
1570  NEXT  J 

1580  FOR  J sO 1 1 . I • *F4 1 1 1 1  TO  U 
1590  RIJ.41*RIJ,4I*I 
1600  NEXT  J 


i  e  i  e  * 
I  620  J 
16  30  F 
1640  F 
1650  f 
I  BBC  F 
1670  . 
1880  t 
1890  f 
1700  > 
1710  f 
I  7?0  ( 
I  7  JO  1 
I  740  I 
1750  1 
1760  I 

I  770 

1 7se 

1  790 
1800 
1810 
I  600 
I  8  32 
1840 
I860 
I860 
1870 
1880 
1890 
1900 
1910 
1920 
1930 
ig40 
1950 
I960 
1970 
I  980 
1990 
2000 
2010 
202C 
2032 
204e 
2252 
2282 
2273 
2083 
2390 

?ioe 
21  10 
2120 
2<30 
2140 
2150 
2160 
2 1  "2 
2180 
2190 
2200 
2210 
2220 
2230 
2240 
2250 


A5*0I  J  .  IUF4  im-i 

PEN  . Unengaged  target*  ond  nieeed  oppe . 

FOR  J*0< 1.21  TO  *5 


RIJ.6URIJ.6UJI 
RI J  ,5URt  J .5U I 
JUJU! 

NE  3  7  J 
NEXT  I 
NEXT  II 

PRINT  u 


act  eng 


Ul  =U-  (R(u-I  .3URIU-I  ,4UN) 

FOR  1=1  TO  Ul 

>>■ R'l. 21. Rt I. 3». RII.41.R5.RII. 51. RU.6I 

IMAGE  3D3X2D3X20IX2D3X2E3X203X2D3X3D 
MsM  MAX  Rli.6! 

NEXT  I 

INPUT  0*  .  . 

REM . Moke  t  Me  gropM.  .axes,  ticks,  ond  volute 

PACE 

MOVE  10.10 
0 RAW  110.10 
MOVE  10,10 
DRAW  10. 100 
X  U  1 00/U 1 
X2=85/M 
FOR  Ul  TO  Ul 
MOVE  17XUI0.I0 
RDRAW  0,-2 

IF  !/5<>INT ( 1/5)  THEN  1040 

PRINT  USING  . HJ“  “2D*  .  I 

NEXT  I 

FOR  J=1  TO  G 

IF  INT IM/151  MAX  UF71J)  THEN  1080 
NEXT  J 

FOR  U0  TO  H  STFP  F71J-I 1 
MOVE  I  0 . I *X2* 1 5 

^F^l/ I5*F7( J- 1 1 1 <>INT 1 1/ I5*F7 t J- 1 1 1 1  THEN  2040 
RMOVE  e.-l 

PRINT  USING  . HHH'"'30"  ■  I 

NEXT  I 
MOVE  0.70 

PRINT  -P  E_R  F.O.R.n.A.N.C.E”; 

MOVE  10.0 

PRINT  TS;'I  UPDATE'; 

1  MOVE  X4XXU9.5,  I  1 
PRINT  “ft" 

I  IF  SUM(CTU0  THEN  2170 
FOR  Ul  TO  SUM  1 C  1  1 
I  MOVE  Hll  UXUW.  5,  I  I 
I  PRINT 

]  gf£T  1  . Plot  unengaged  tdrgete  end  nleeed  epponun;  t  ies 

1  MOVE  XU9.RH  , 6 1  *X2*  1 5 
1  FOR  U2  to  Ul 
)  DRAW  J*XUS.RII.6l»X2»t5 
1  NEXT  I 
)  FOR  Ul  TO  Ul 
I  MOVE  IXXUS,R<1.5)*X2«I4 
!  PRINT 
3  NEXT  I 
3  ENO 


-  I***'*  ’■  V 
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APPENDIX  D 

DATA  ANALYSIS!  PROGRAM  VARIABLES 


D-0 


VARIABLE 


NAME 

DEFINITION 

TYPE 

A$ 

Target  speed  designator (F,  M,  or  S) 

String 

A1 

Target  bearing(degrees) 

Array (N) 

A5 

Target  entry  time:  plus  1000 Chit) 
or  2000 (killed) 

Array (N) 

A8 

Target  track  number (TN) 

Array (N) 

c$ 

Outcome  designator(hit  or  kill) 

String 

Cl 

No.  of  hits  (fast,  medium,  slow) 

Array (3) 

C2 

No.  of  kills  (fast,  medium,  slow) 

Array (3) 

D 

Code  for  tempo  of  operations 

Simple 

F 

No.  of  updates  until  optimal  launch 

Array (3) 

F$ 

Input  for  file  to  be  analyzed 

String 

F4 

No.  of  updates  until  hit  occurs 

Array (3) 

F5 

Target  speeds 

Array (3) 

F6 

Range  at  entry  time 

Array (3) 

F7 

Scale  for  ordinate  in  M0  graph 

Array (6) 

H 

Updates  on  which  hits  occurred 

Array 

I 

Index  in  for-next  loop 

Simple 

11 

Index  in  for-next  loop 

Simple 

J 

Index  in  for-next  loop 

Simp  -.e 

J1 

Counter  for  missed  opportunities 

Simple 

K 

Update  at  which  kill  occurs 

Simple 

K4 

Update  for  first  launch  opportunity 

Simple 

K5 

Last  update  prior  to  a  hit 

Simple 

K9 

Average  distance  for  kills 

Simple 

L 

Length  of  L$ 

Simple 

L$ 

Player’s  launch  history 

String 

L7 

No.  of  inflight  launches 

Simple 

L8 

No.  of  splashes 

Simple 

L9 

No.  of  good  launches 

Simple 

M 

Maximum  value  of  MO  by  update 

Simple 

N 

Total  no.  of  targets 

Simple 

D-l 


r  ■» 


VARIABLE 

KAME 


DEFINITION 


TYPE 


NO 

No.  of  targets  of  each  speed 

Simple 

Q 

Table  of  target  and  launch  information 

Array(N,5) 

Q$ 

Text  for  game  parameters 

String 

Q6 

Lag  score 

Simple 

R 

Table  of  target  status  by  update 

Array(U,6) 

R1 

Skill  rating 

Simple 

R5 

No.  of  targets  remaining 

Simple 

S 

Sum  of  squares  for  lags 

Simple 

s$ 

Player  I.  D. 

String 

T$ 

File  number  designator 

String 

T1 

Code  for  update  extracted  from  L$ 

Simple 

12 

Code  for  track  number  extracted  from  L$ 

Simple 

U1 

Last  update  of  game 

Simple 

W 

Table  for  track  number  lookup 

Array (9 9) 

X 

Sum  of  lags 

Simpl e 

X$ 

Launch  Information  extracted  from  L$ 

String 

XI 

Temporary  storage  for  kill  computation; 
Unit  for  abscissa  in  MO  graph 

Simple 

X2 

Temporary  storage  for  kill  computation; 
Unit  for  ordinate  in  MO  graph 

Simple 

Y 

Summary  table  of  lags,  inflight  launches, 
and  splashes  by  target  speed 

Array (3, 4) 
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